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ABSTBACT 

A vealth of instructioti '-1 information is intrinsic to 
standardized achievement tests. Most practioners fail to realize the 
availability or potential of item analysis information or are unable 
to utilize such information appropriately* Achievement tests have not 
fully accomplished their purposes if the iformation derived stops at 
just test scores without the tests affording the information 
concerning pupils' responses to the item^ in the test battery. Such 
information can then provide a real basic upon vhich teachers can 
build an effective instructional program in the areas tested. This 
paper provides a rationale for an unconv^^ntioual yet meaningful item 
analysis and describes the procedures vhich will lead to greater 
implementation of item analysis data. (Author) 
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Title: ITEM ANALYSIS DATA - AN UNCONVENTIONAL APPROACH TO .. . 
INTERPRETATION AND USE , - 

Author: Jules Grosswald 

Division of Testing ; 
The School District of Philadelphia 

A wealth of instructional, information is intrinsic to 
standar'^ized achievement tests. Most practioners fail to realize 
the availability or potential of item analysis information or are 
unable to utilize such information appropriately. 

Achievement tests have not fully accompl i stied their 
purposes if the information derived stops at just test scores without 
the tests affording the information concerning pupils* responses to 
the items in the test battery. Such information can then provide a . 
real basis upon which teachers can build an effective instructional 
program in the areas tested. 

This paper provides a rationale for an unconventional yet 
meaningful item analysis and describes the procedures which will, lead 
to greater implementation of item analysis data. ^ 
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SUMMARY 

— n ' — 
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Objectives i 

This paper attempts to demonsti-ate the wealtK of i nfitruct i ona 1 information 

that can be obtained by measurement through the use of standardized achievement tests. 

Most practioners fail to realize the ai'ai labi 1 i ty or potential of item analysis 

|l 

information or are unable io utilize such information appropriately. 

i i ■ ■ 

The paper will not cnly proviBje a rationale for an unconventional yet 
meaningful item analysis but vfll als [describe the procedures which will lead to 
greater implementation of iter analysis! data. 



Methods and Techniques I i 

i i 

School item analysis reports for each grade tested are utilized. The item 

I • ; I 

analysis report presents a new dimension in test reporting, because for the first 

■ ! i ' 

f i 

time full advantage can be taken of computer capabilities to produce a report that is 

^ I 
meaningful, relatively easy to interpret and use in planning and implementing programs 

of instruction designed to overcome weak performance in those areas indicated by the 

statistical data! for the school. ! 

The report presents, in skill! order sequence, pupi. 1 performance on each test 

1 

item pertaining to each of the skills bsing measured* In such a format, the report 

makes it possible for the user of test||clata to make comparisons on a skill level basis 

It permits ready and easy answers to the following questions: 

1. Where are the pupils* strengths? 
.2. Where are the pupil s * weak|iesses? 
3. How did pupil performance! in the school compare with school 

system*s performance at ea::h grade level 
^. How did pupil performance ! i n the school compare with national 

performance at each gradej level? 

5. How did pupils in the school perform on relatively difficult items? 

6. How did pupils in the schoDl perform on items of average difficulty? 

7. How did pupils in the school pe /form on relatively easy items? 



\ Item analysis data are useful mainly in analyzing the characteristics of 
local test performance. However, for item data to be useful, two kinds of information 
are necessary for comparison. The first is the school *s results which are to be 
recorded in work table booklets. This is the percent of pupils in the school who 
answered an item correctly. The second kind of information necessary for comparison 
is the discrepancies which result when comparisons are made to national Ttem norms. 

The data organization procedure requires two documents, and I tern Analysis 
Report and a set of Work Tables for the reduction of the data to a usable format. 

The item analysis report should use a consistent format in reporting pupil 
performance for all grade levels tested. For each subtest of the battery, the 
following information should be provided: 

1. the numbers of the items In the subtest for which data are given 

2. the percents of pupils in the grade in the school who completed 
the entire battery and who answered each Item correctly. 

3. the percents of pupils in the publijher*s standardization population 
who completed the entire battery and who answered each item correctly. 

A. the percents of pupils in the grade . in the city who completed the 
entire battery and who answered each item correctly. 

The work tables are used in conjunction with the school item analysis 
report. An example for a reading comprehension test will Illustrate the use of the 
work tables. Each table contains (l) sM 1 1 s classification for the test, C2) the 
national midyear norms for item performance, (3) the system end-of-year norms for 
item performance, and (A) a place to record the actual percent correct" for each 
item for the school. Numbers from the item analysis reports are to be copied, 
accurately, into the work table for each reading skill.. 

After the percents have been copied and checked proceed to compute' the 
differences (deviations) between performance in the school and performance in the 
national sample population and to enter the results in a 'Difference* column. When 
the differences have been computed and checked find the 'sum of the + differences* 
and the 'sum of the - differences' as shown in the table. 



Directly belov; each skill area work table, there appeals a rectangle 
outlining a step-by-step procedure for computing the average difference of all the 
items measuring the particular skill involved. 

On front of the work tables booklet, there appears a summary table of the 
average differences (deviations) for all skill areas for which averages were computed. 

Assuming, therefore, that a decision has been reached concerning the skill 

areas which evidence the greatest need for attention, we next study the work tables 

for more specific information and a direction to try to pursue in classroom 

instruction. The third column of the table labeled "Difficulty of Item" attaches 

to each item a symbol describing the level of difficulty based on how the national 

population performed on each item. 

E - an easy item, answered correctly by 65% or more of 
the national population and therefore within a 
reasonable range for most pjpils to master. 

A - an average item, answered correctly by between 35% 
and of the national population and therefore 
capable of mastery by those pupils in the middle of 
the distribution. 

H - a hard item, answered correctly by 3kZ or fewer of 
the national population and therefore an item 
designed to challenge only the more able pupils in 
any school s i tuat ion . 

It would seem reasonable that those Items classified as "Average" or 
"Easy" where the school's performance level fell below both National and system 
performance suggest weaknesses and need the most immedia*te attention. The 
"Difficult" items, at this level, should be considered but not with equal emphasis. 

Those items which signify school performance between the national sample 
and the city, in most school situations, probably may reasonably be cpnsldered as 
"strengths" which can be effectively used as aids in improving performance. Similarly, 
those items in which school performance surpassed both the national sample and the city 
should definitely be regarded as distinct "strengths" in performance and the skills 
Involved can be used to build upon and overcome those skills reflecting weakness. 
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Data Source 

The development and implementation of this procedure for meaningful item 
analysis was done in the School District of Philadelphia with the cooperation of 
the Houghton Mifflin Company, Publisher of the Iowa Tests of Basic Skills. The 
complete paper will contain examples and illustrations of the procedure. ^ 

Conclusions 

Users have indicated that this item analysts approach provides the kind of 
assistance which makes it possible for each school to organize a concerted attack 
on the specific problems within the school which will result in a constructive 
program of learning activities within the framework of chs local curriculum 
emphases leading to a general improvement in the achievement levels of the pupils 
throughout the system. 

Educational Importance of This Papo.r 

The use of nationally-standardized, norm-referenced achievement testing is 
under attack because such testing is perceived by many practioners as having very 
low instructional yield. This paper may help to change that perception to a more 
favorable one regarding helpfulness and utility. 



ERIC 



